Skip to content

Local test execution setup#63

Open
WilliamRoebuck wants to merge 1 commit intoeclipse-score:mainfrom
etas-contrib:feature/system-testing-setup
Open

Local test execution setup#63
WilliamRoebuck wants to merge 1 commit intoeclipse-score:mainfrom
etas-contrib:feature/system-testing-setup

Conversation

@WilliamRoebuck
Copy link
Contributor

Add a basic setup to run system-level tests for lifecycle on the host machine. Resolves eclipse-score/score#2403

The minimal example shows a GTEST executable launched by LM that tests simple state transitions.

@github-actions
Copy link

github-actions bot commented Feb 5, 2026

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.4.2) and connecting to it...
INFO: Invocation ID: f407cdac-1305-4b89-9168-b5067915d1a5
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
WARNING: For repository 'rules_python', the root module requires module version rules_python@1.4.1, but got rules_python@1.8.3 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'rules_cc', the root module requires module version rules_cc@0.2.14, but got rules_cc@0.2.16 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'aspect_rules_lint', the root module requires module version aspect_rules_lint@1.5.3, but got aspect_rules_lint@2.0.0 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'download_utils', the root module requires module version download_utils@1.0.1, but got download_utils@1.2.2 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'googletest', the root module requires module version googletest@1.17.0.bcr.1, but got googletest@1.17.0.bcr.2 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
WARNING: For repository 'score_rust_policies', the root module requires module version score_rust_policies@0.0.3, but got score_rust_policies@0.0.5 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (34 packages loaded, 9 targets configured)

Analyzing: target //:license-check (87 packages loaded, 11 targets configured)

Analyzing: target //:license-check (140 packages loaded, 2614 targets configured)

Analyzing: target //:license-check (149 packages loaded, 5596 targets configured)

Analyzing: target //:license-check (155 packages loaded, 7768 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

Analyzing: target //:license-check (160 packages loaded, 7817 targets configured)

INFO: Analyzed target //:license-check (165 packages loaded, 9955 targets configured).
[5 / 16] [Prepa] Writing repo mapping manifest for @@score_tooling+//dash/tool/formatters:dash_format_converter [for tool] ... (2 actions, 1 running)
[13 / 16] JavaToolchainCompileClasses external/rules_java+/toolchains/platformclasspath_classes; 0s disk-cache, processwrapper-sandbox
[14 / 16] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 24.341s, Critical Path: 2.60s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@WilliamRoebuck WilliamRoebuck force-pushed the feature/system-testing-setup branch from b8443f6 to 905218a Compare February 5, 2026 12:06
@github-actions
Copy link

github-actions bot commented Feb 5, 2026

The created documentation from the pull request is available at: docu-html

Copy link
Contributor

@pawelrutkaq pawelrutkaq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

@NicolasFussberger
Copy link
Contributor

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score.
The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.

I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development.
So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.

This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

@pawelrutkaq
Copy link
Contributor

pawelrutkaq commented Feb 11, 2026

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score. The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.

I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development. So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.

This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

itf can run docker, I am bit wordied that not we doing simply third thing becasue its faster to do some few python code. I will talk to Piotr, but we shall still not merge it to main before release as first we need other fixes - i dont see this to hurry

@MaciejKaszynski
Copy link
Contributor

This shall be synced with @PiotrKorkus hwoto approach tests. There are score tools for it. We shall make it consistent.

I had a brief chat on slack with Piotr and as far as I understood there are 2 options in Score. The framework from tooling repo which only allows local execution and the ITF framework which allows execution on qemu machine.
I believe, since the launch manager has some qnx-specific functionality (like e.g. security policies), that testing on QNX is a must -have. But always running tests in QEMU is also a pain, which slows down development. So the idea was to have the best of both worlds, to allow just executing tests locally (on linux), but also allow to run the tests in qemu using ITF. So this PR adds a minimal abstraction that lets you start something locally or via ssh on qemu machine.
This is only the first step to get some basic smoke tests running. Maybe we can agree on a first solution just to get some integration tests working soon. I feel like there is no consistent solution in SCore at this point.

itf can run docker, I am bit wordied that not we doing simply third thing becasue its faster to do some few python code. I will talk to Piotr, but we shall still not merge it to main before release as first we need other fixes - i dont see this to hurry

So we are also working on a ITF integration and that is where the real testing is done (Draft PR #75). Right now we are using QEMU targets but I had a brief talk with an ITF person, and the goal seems to be for different targets to be transparent to the test, meaning that using docker vs QEMU can be changed with just a argument.
(I know currently there is some re-writing happening in ITF and the APIs aren't final. There are still some issues for example with docker vs QEMU as running a command on the target requires different python code. But it sounds like at some point it would be the same.)

This PR allows for native testing which is just a improvement for developers. I don't think there is anything extra needed to allow for this. There is a wrapper here for the testing framework which is a good idea anyways to have a more consistent interface. The PR will just add an implementation for the interface for native support and the previously mentioned branch would add an implementation for ITF support.

- Local execution
- Launching LM daemon with a config
- GTEST processes launched by LM with test results parsed
- Test timeout
- Scaffolding for platform independent python scripts
- Running LM without root
- setgroups() is not attempted if there are no supplementary group ids
- Minor cleanup in ProcessGroupManager::initialize()
@WilliamRoebuck WilliamRoebuck force-pushed the feature/system-testing-setup branch from 7780751 to 2318fe0 Compare February 19, 2026 10:42
@pawelrutkaq pawelrutkaq dismissed their stale review February 19, 2026 10:57

Relaxing request of change - still on position we shall use itf and nothing custom unless we can prove itf is too big overhead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local test execution

4 participants

Comments